List of Flash News about model evaluation
| Time | Details |
|---|---|
|
2026-01-23 00:08 |
Anthropic Releases Petri 2.0 Open Source AI Alignment Audits With Eval Awareness Countermeasures and Expanded Seeds
According to @AnthropicAI, the company released Petri 2.0, an open source tool for automated alignment audits that adds countermeasures against eval awareness and expands seeds to cover a wider range of behaviors after adoption by research groups and trials by other AI developers, with no crypto or token integrations disclosed, source: https://twitter.com/AnthropicAI/status/2014490502805311959. |
|
2025-10-28 23:41 |
Stanford AI Lab Launches SLP-Helm Pediatric Speech AI Benchmark: Bias Findings and What Traders Should Note
According to @StanfordAILab, the lab released SLP-Helm, a benchmark that tests how AI models diagnose pediatric speech and reveals promise, pitfalls, and bias; source: Stanford AI Lab X post on Oct 28, 2025 and Stanford AI Lab blog. According to @StanfordAILab, millions of children face speech disorders and few receive timely care, providing the clinical context for evaluating diagnostic model performance; source: Stanford AI Lab X post on Oct 28, 2025. According to @StanfordAILab, further details are provided on the Stanford AI Lab blog for reviewing the benchmark’s tests and findings; source: Stanford AI Lab blog referenced in the X post on Oct 28, 2025. |
|
2025-02-05 16:51 |
Gemini 2.0: Superior Price/Performance Model Outshines GPT-4o
According to @SullyOmarr, Gemini 2.0 is currently the best model for price/performance ratio. Evaluations suggest it outperforms GPT-4o, offering significant cost benefits by being approximately 20 times cheaper. This makes it a recommended choice for users not focused on coding-intensive tasks. |